Integrated Medical Workflow for Augmented Reality Applications

Augmented Reality (AR) and Virtual Reality (VR) have the potential to improve surgical interventions and image-guided therapy by providing clinicians with interactive three-dimensional visualizations in all phases of treatment. However, experimental AR research applications typically focus on a single medical procedure and do not attempt to cover the whole clinical workflow from preoperative to intra-operative and post-operative support. In the scope of the EU-funded Marie Curie research project ARIS*ER (Augmented Reality in Surgery), we are investigating a more complete workflow support with AR technology. This paper discusses a component-oriented software architecture which specifically aims at addressing the needs of medical AR, in particular strategies for ensuring uniform application quality throughout the medical workflow. We discuss the management of off-line and on-line image acquisition methods and methods for consistent management of modeling, processing and visualization of medical datasets throughout the medical workflow. Finally, we present three different medical application examples which illustrate the use of the framework. CR Categories: D.2.6 [Programming Environments]: Graphical environments—Integrated environmentsInteractive environmentsProgrammer workbench H.5.1 [Multimedia Information Systems]: Artificial, augmented, and virtual realities— [I.3.7]: ThreeDimensional Graphics and Realism—Virtual reality